INAOE at QA@CLEF 2008: Evaluating Answer Validation in Spanish Question Answering

This paper introduces the new INAOE’s answer validation method. This method is based on supervised learning approach that uses a set of attributes that capture some lexical-syntactic relations among the question, the answer and the given support text. In addition, the paper describes the evaluation of the proposed method at both the Spanish Answer validation Exercise (AVE 2008) and the Spanish Question Answering Main Task (QA 2008). The evaluation objectives were twofold. One the one hand, evaluate the ability of our answer validation method to discriminate correct from incorrect answers, and on the other hand, measure the impact of including an answer validation module in our QA system. The evaluation results were encouraging; the proposed method achieved a 0.39 F-measure in the detection of correct answers, outperforming the baseline result of the AVE 2008 task by more than 100%. It also enhanced the performance of our QA system, showing a gain in accuracy of 22% for answering factoid questions. Furthermore, when there were evaluated three candidate answers per question, the answer validation method allowed increasing the MRR of our QA system by 40%, reaching a MRR of 0.28.